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The materials in this manual are designed to help 
workshop leaders prepare for and present a workshop for school board 
members on program evaluation. The board's role in program evaluation 
is primarily to set. the goals and philosophy of the district, to 
establish policy for program evaluation, to examine the congruence 
between the district's philosophy and its programs, and to make 
program decisions_.on the basis of this information. The workshop s 
four basic segments justify school board involvement in the 
curriculum, cover the school board's role in program evaluation, 
discuss the use of evaluation information in decision-making, and 
apply the workshop lessons to individual district situations. The 
workshop manual consists of six sections: an introduction; three 
sections on planning, presenting, and evaluating the workshop; a 
selection of resource materials; and a booklet for workshop 
participants. The- discussion of planning covers the focus and scope 
of the workshop, its organization, and familiarization with relevant 
materials. The presentation section includes sequential descriptions 
of workshop activities and instructions for conducting those 
activities. The resource materials provided include a paper on 
evaluating instructional programs, and masters from which 
transparencies can be reproduced for projection at the workshop. 
(Author/PGD) 
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These materials have been produced as part of "Keys to School Boardsmanship," 
a project to develop new materials in boacdsmanship education for local school 
board members. It is a joint effort among: 

o The Association of Alaska School Boards' 

o The Idaho School Boards Association 

The Montana School Boards Association 

The Oregon School Boards Association 

o The Washington State School Directors* Association 

o The National School Boards Association 

o The Northwest Regional Educational Laboratory 

Copyright (c) 1982 by the Northwe*^ Regional Educational Laboratory. Rights 
to publication are held by exclusive agreement by the National School Boards 
Association* Copying or duplication by any means (print, electronic, etc.) is 
prohibited, except by written authorization by the National School Boards 
Association. 

These works were developed under Contract #400-80-0105 with the National 
Institute of Education, United States Department of Education. However, the 
content does not necessarily reflect the position or policy of that agency, 
and no official endorsement of these materials should be inferred. 
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Prologue : 



This manual is one in a series of thirteen developed by the Northwest Regional 
Educational Laboratory (NWREL) as part of the "Keys to School Boardsmanship rt 
project. 

The workshop programs resulting from the project are intended to help 
strengthen local school boards through continuing education. 

The workshops in the series have been thoroughly tested by NWPEL staff and by 
the staff of state school board associations in Alaska, Idaho, Montana, Oregon 
and, Washington. These programs have been found to be extremely useful as 
tools for strengthening continuing education services to school board members. 

The "Keys to School Boardsmanship" materials are now being published and 
distributed by the National School Boards Association (NSBA) . In addition, 
the NSBA is developing a national center designed to assist state associations 
to make the best use of the material. 

Thirteen manuals in the series now available from NSBA include: 
o Board/Administrator Relations 

o Building Bridges: School Board Political Roles 

o Communicating with the Community 

o Conflict: Alternatives to Blowing a Fuse 

o Effective School Board Meetings 

o The Educational Management Team 

o Policy Development 

o Policy is Power 

o Program Evaluation: School Board Roles 
o What Do School Boards Do? 
o School Board Self-Assessment 

o School Improvement: A Tune the School Board Can Play 



o 



Teamwork: The Board and Superintendent in Action 



SECTION 1;, INTRODUCTION 



Purpose and Rationale 

i 

Program evaluation is a systematic collection of information about a program 
which includes a judgment of worth and recommendations for improvement of and 
decisions about the program. The primary purpose of program evaluation is not 
to prove but to improve , Of course, decisions do have to be made to cut back 
or discontinue programs. Even then, program evaluation can help insure that 
the decision is based on good information 1 rather than opinion. Evaluation is 
an important part of a well functioning program. 

Evaluation of programs should be systematic, continuous and monitored the 
board • As the board looks at the programs in the district, the cycle enlarges 
to include other steps. The board can look at a program through evaluation to 
see if it is consistent with the goals the board has set for the district. It 
can then monitor the stages of implementation and evaluation, and then use 
that evaluative information to help you make decisions which will result in 
revision or improvement of the program to more closely reflect the goals and 
philosophy of the district. The board's role, then, is primarily to set goals 
and philosophy of the district, set policy for program evaluation, request 
evaluation, use evaluation information for its program decisions and examine 
its congruence between programs and philosophy. 

Emphasis is placed on the ways in which program evaluation information can be 
used to help board members in specifying, the basis of decisions, communicating 
with the community and becoming more knowledgeable about their district 
program. 

The roles of the board members in program evaluation stressed in the workshop 
are to: ** 

o Ask questions — description of program and impact of program 

o Use evaluation information to help them make decisions about programs 
and policy 

o Set policy to insure ongoing evaluation within district 

The workshop package includes a participant's workbook, a set of 
transparencies, a workshop leader's guide, several written curriculum 
situations, a resource list, other handouts which may be used, and a workshop 
critique form. The workshop is divided into four basic segments: 

1, Introduction and justification for school board members to be 
involved in curriculum 

2, Determining what the roles of the school board and the educators are 
in program evaluation 



3. Using program evaluation information in school board decision making 
(identifying decisions and critical questions on specific curriculum 
situations) 

4. Applying the prioritizing exercise to their own unique school 
district situations 

The Program Evaluation workshop is developed around nine learning activities . 
Each activity contains a variety of instructional and experiential activities 
that are designed to help the participant achieve specific objectives. 

Goals and Objectives 

The goals of the Program Evaluation workshop are: 

o To introduce board members to the notion that evaluation judgments 
are always being made 

o To demonstrate that the most effective judgments are the result of a 
planned, systematic cycle of evaluation 

The objectives of the Program Evaluation workshop are: 

o To demonstrate that school board members can play an important role 
in program evaluation 

o To define program evaluation as a two-step process: (1) information 
collecting and (2) judging 

o To demonstrate a procedure that might be used by school board members 
in program evaluation 

o To identify and practice a three-step decision making process that 
can be used by a school board in program evaluation. The steps are: 
(1) identify the decision to be made, (2) list the questions that 
need to be asked and (;\) select the most important questions. 

o To explore the school boards role in program evaluation 

o To develop a series of questions that board members can take back to 
their school districts about program evaluation 

Organization of the Leader's Guide 



The Leader's Guide contains the basic directions and materials you will need 
to plan the workshop. Any additional information or experience that you can 
bring to the workshop will serve to strengthen your presentation. 



Planning the Workshop 

Section 2 contains information about preparations the 
leader needs to make before presenting the workshop. 
Included are hints and suggestions about: 



o 


Focus and scope 


o 


Objectives and time estimates 


o 


Becoming familiar with the material 


o 


Reviewing Section 3 


o 


Skills needed 


o 


Organizing for learning 


o 


Grouping of participants 


o 


Use of transparencies 


o 


Warnings 


o 


Equipment needed 



Presenting the Workshop 

Section 3 includes a step-by-step description of the 
workshop procedures, along with the time allocations and 
the required resources. In addition, you will find: 

o A sequence of workshop activities 

o Specific instructions and references to participant 
materials and audio and visual aids 

o A leader focus for each workshop activity 

* 

o Helpful hints about workshop activities 
Evaluating the Workshop 

Section 4 contains the procedure used for evaluating the 
workshop. 

o The background of the development and testing of this 
workshop 



< 



Section 5: 



Resources 



This section contains reading material which the leader 
will need to become familiar with before presenting the 
workshop. Included in this section are: 

o ( Concept papers 

o Background reading and references 

o Further explanation of ideas developed in the workshop 



Section 6: 



Participant Materials 
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SECTION 2: PLANNING THE WORKSHOP 



Focus and Scope 

The Program Evaluation workshop is composed of nine instructional activities, 
each with- a set of unique objectives, it is possible to redesign the workshop 
to emphasize specific objectives or'to raeet-gisjen time requirements; however, 
a word of caution is in order. The workshop wiilM?e less successful if in the 
redesigning the time allocated for group discussions> N interactions and 
simulations is eliminated. This program should not be tinrned into" a lecture 
because of time limitations. 

Time Estimates and Activities 

The following outline presents the activities and the approximate time needed 
for each. 



Total Time: PROGRAM EVALUATION WORKSHOP 

2 hrs., 45 rain. 



Time 


Activity 






15 


rainu tes 


Activity 


1: 


Introduction and Objectives 


20 


minu tes 


Ac tivity 


2: 


Defining the Learning Program 


25 


minutes 


Activity 


3: 


Evaluation Exercise 


25 


minutes 


Activity 


4: 


What is the Board's Role? 


5 


minutes 


Activity 


5: 


The Board's Role 


15 


minutes 


Activity 


6: 


Use of Evaluation Information 


10 


minutes 


Activity 


7: 


Review and Legal Basis 4 


40 


minutes 


Activity 


8 


Questions 


10 


minutes 


Activity 


9: 


Summary 


10 


minutes 


Activity 


10: 


Evaluation 



Becoming Familiar With the Materials 

This workshop has been designed around a very specific notion about the school 
board's role in program evaluation. The papers presented in Section 5 of this 
notebook were developed, to provide you with the background, research and 
theory upon which this workshop is built* They do not represent the last word 
on the subject, and any additional research .and reading that you can bring to 
the subject will strengthen your presentation. 
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Reviewing Section 3; Presenting the Workshop 



Section 3 of the Leader 1 s Guide is a step-by-step outline for presenting the 
workshop. There are a number. of "cues" that have been used to assibu you in 
the presentation; These are as follows: 

o Activity 

Each unit of instruction is identified by an activity number and 
title, both of which are underlined, i.e., Activity 2; Defining the 
Learning Program . Each activity stands as a unique unit of learning 
with its own objectives^ participant activities and basis of 
knowledge. The various activities are related in that they are all a 
part of a larger common concept, in this case, program evaluation. 
(See Figure la.) 

o Leader Focus 

Immediately underneath the activity number and name there is a 
heading, Leader Focus , that is also underlined. .The Leader Focus 
provided you with a "mind set," a "feeling," an "attitude" or an 
"anticipatory set" that you should attempt to establish in this unit 
of instruction. The Leader Focus flight be defined as the "affective" 
objective for the unit. (See Figure lb.) 

o Time 

The total time required for each activity will be located at the top 
left hand column of the first page of the activity. Additional 
divisions .of time within' the activity will also be noted in the left 
hand column. (See Figure lc.) 

o Materials 

The right hand column of each page will list the materials, 
transparencies, workbook references or other activities for 
instruction that need to be brought to your attention. The 
references will be noted in the right hand column next to the 
appropriate point of instruction. (See Figure Id.) 

The middle section will use three cues. They are: 

o Leader Instructions 

— Subpoints 
o Lecture tte 

— Subpoints 

O CAPITALIZED WORDS 
— 1 Subpoints 
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Leader Instructions 



The cue, Leader Instructions , alerts you, the workshop leader, to 
something you must do, say or cause to happen. Each of the following 
bullets "o" (see Figure le) are additional sequential cues to the 
leader. 

o Lecturette 

The cue, Lecturette , alerts you that at this point in the workshop 
you are expected to provide a knowledge base or lecture about a given 
subject. The main points in the* lecture will be outlined in the 
following paragraphs until you reach a new cue. You will also find 
in Section 5, Resources, papers that will provide additional 
background for the lecture. It is best to make the main points in 
your own language or words. Do not read the lecturette . (See 
Figure If.) 

o CAPITALIZED WORDS 

The cue of CAPITALIZED WORDS within the general framework of the text 
alerts you to the advisability of reading the exact words to the 

participants . The exact words in this case will help direct all the 

participants through the planned learning activity. (See Figure lg.) 

o The cue " — " is used to alert you to subpoints that may be used to 
add to or clarify a thought or statement. (See Figure lh.) 

o Helpful Hints is a cue that will appear at the end of some 

activities. Its intent is to cue you into something that will help 
make the workshop more successful. (See Figure li.) 

o A solid line across the entire page will be used to indicate the end 
of the activity. (See below and Figure lj.) 



Figure 1 on the following page is a visualization of what might occur in 
one unit of instruction. 



/ 



FIGURE 1 



TIK" 



(c) 



Total 
Time: 
30 rain. 



5 rain. 



20 min. 



(e) 
(e) 
(e) 
(h) 
(h) 



(g) 

o 



Lecturette: 



(f) 
(f) 



Ask them, IP THE SCHOOL BOARD IS IN CHARGE, WHY 
DON"T YOD DO IT? 

(g) 



(subpoints) 



(h) 



Helpful Hint : Move this section very fast... 



(d) 

MATERIALS 



Activity 2; Defining the Learning Program (a) 



Leader Focus ; (b) 

o Leader Instructions : 

o 

o 

— (subpoints) 

— (subpoints) 



(d) 

Trans. PE 1 



d) 

Workbook 
Page 2 



(j) 
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Skills Needed for the Workshops 



Before you make a decision to use these mater ials, you might take a few 
minutes and think through four interrelated decisions you should consciously 
make. All workshop leaders or teachers make the same decisions. The only 
difference is that a few understand they are making the decision, and the 
others do it by the "seat of their pants ," or as Madeline Hunter says, "like 
egg on a wall you may or may not hit the spot." The decisions are: first, 
what are the needs of the participants; second, how much time do you have to 
work with the participants (one time only for one hour, several times for one 
hour, one time for four hours, etc); third, what extent of knowledge do the 
materials represent; and fourth, what are your skills in workshop leadership 
and competencies in the subject matter? 

Participant Needs 

In any workshop the participants will come to the program with a variety of 
needs, beliefs and styles. Most adults learn best about that which they can 
immediately use. However, within that statement there is a range of 
participants; there is a group that just wants to learn about the idea, still 
another group that wants to develop skills and a third group that wants to 
immediately apply the information. Of course, it takes less time to expose a 
person to an idea, more time to teach participants a skill, and even more time 
for them to learn how to apply and use the skill. 

At the same time, many participants come to a conference solely for the 
opportunity to socialize, discuss current issues and to strengthen and enlarge 
their interactions and acquaintance with other school board members. The 
materials in this workshop were designed to meet these "inclusion" and 
" belong ingness" needs through controlled small group discussions and 
interactions. The word "controlled" is used because these same adults have a 
need to appear to be in control, and at the same time, they want the workshop 
to be in control, and be of value and present a worthwhile learr ing experience. 

Small group interactions are an important foundation of the design because 
they also encourage "feedback." Adults need the opportunity to apply and try 
out new knowledge. Adults need to know where they stand. They need an 
opportunity to test new ideas and to make mistakes. The small groups allow 
the freedom and create the control needed to reinforce described learning 
outcomes and to help mistakes become valuable learning opportunities. 

The use of small groups provides a balance between the "expert" workshop 
leader and the "experience-based" school board member. The small groups 
create an opportunity for the school board member to test his/her experiences 
with the ideas of the presenter. 

Any attempt to reduce the small group interactions in this workshop in order 
to "give more information" will surely " gut. " the basis of the design and 
ignore the needs of the participants. 



Time 



The critical element in learning is time. The biggest mistake the workshop 
leader can make is to attempt to build skills or force technical application 
of knowledge without enough time. Time relates to learning* You must relate 
the time available to the needs of the participants* The critical decision is 
whether you have enough time to meet the participants 1 needs and the 
objectives. A workshop designed to develop skill will take more time than one 
designed to expose the participants to an idea. 

Level of Materials 

The research and development of the concepts and materials in this workshop 
centered on three levels. The first level was identified as the "introduction 
level," defined as representing enough of the generalized concepts in the 
subject area to present a current overview of the subject matter. The second 
level was identified as skill building, which was described as representing 
the basic skills in the subject matter. Finally, the third level was 
identified as the technical assistance level, representing an expansion of the 
basic skills to include application, analysis, synthesis and evaluation skills. 

The materials in this workshop were developed to be used at an exposure or 
skill building level. The skills that are represented in the materials are 
specific and by no means represent a complete view of the knowledge on the 
subject, if the workshop leader or association representative is expected to 
work at a technical assistance level with an individual school district, 
he/she would snrely be expected to bring additional knowledge and skills to 
the experience. 

Leader Knowledge and Skill 

A leader with a minimum amount of skill and knowledge should be able to 
present the workshop at both an introduction and skill building level by 
following the suggested steps in the Leader's Guide. In this case, the 
workshop leader should stay very close b to the suggested outline. After 
presenting the workshop several times, the leader may then be ready to create 
his/her own choices within the materials. If, on the other hand, the workshop 
leader is an experienced workshop leader with an extensive knowledge in the 
subject area, the materials in the Leader's Guide- may well serve only as a 
reference or guide. In this case, the leader may choose to alter the material 
to meet his/her own style or workshop objectives. 

The concept papers and other material in the reference section of this 
Leader's Guide provide a ready reference for the growing leader who wishes to 
expand beyond the structured material. Just about any material including the 
most complex mathematical problems can be delivered with a structured, 
mechanical knowledge level approach. The growing, creative workshop leader 
will move to a higher level of understanding — application, analysis, analysis 
of relationships and analysis of organizational principles. At this point, 
the leader will be in a better position to help individual school boards on a 
technical assistance level. 
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The question for the leader is, are you going to throw egg at the wall or are 
you going to practice what you preach— quality workshop, leadership and quality 
representation for local control? 

In any case, all workshop leaders are advised to review all the material, the 
transparencies and the workshop exercises before making the first presentation 
in public. 

Workshop Difficulty 

The Program Evaluation workshop systematically defines the role of the school 
board in evaluating the district's programs. This is an easy program to 
present; however, you should bring some knowledge of program evaluation to the 
workshop. The materials with a medium level of knowledge from the leader will 
serve as an outline for skill building and technical assistance. 

Organizing for Learning 

The Program Evaluatio n workshop is designed to bring together the personal 
experiences of the participants with the new experiences and information 
presented in the workshop. Consequently, the most effective way to use the 
materials and to organize the workshop is to create an environment that 
facilitates interaction of the participants with the new information. 
Conversely, the least effective way to use the materials is to lecture to the 
participants. 

The most appropriate arrangement for the room is one with tables to 
accommodate seven to ten people organized in such a manner that all th§ 
participants can see a central projection area. 

If small group, arrangements are not possible, attempt to provide for 
arrangements where the people can move around for worksessions. 

As workshop leader, you are encouraged to adjust, add or remove parts of this 
workshop to meet your style and the participants 1 needs. It won't be yours 
until you change it. 

Depending upon your goals, the groups can be either organized around "home 
groups" or "stranger groups." 

Home groups give the participants an opportunity to use live, meaningful 
examples from their own district. However, it limits the inclusion of new 
ideas, limits workshop communication, and allows the typical dominating 
superintendent or board member to continue to dominate. 

Stranger groups provide an opportunity to increase communication, between 
school boards, which tends to facilitate sharing new ideas. "This, of course, 
may be good or bad. The stranger groups do allow board members to "rehearse" 
new roles without feeling as self-conscious or threatened. 



--- - ■ ■ ■ - - ■ ■ ■ — ■ ■- -- 



Use cf Transparencies , 



Transparencies in this workshop serve a rather unique function, the same 
function for the workshop leader as "cue" cards do for TV broadcasters. The 
transparencies have been designed to free you from the "written page" of the 
Leader 1 s Guide. The transparencies carry in an abbreviated form the main 
points of a lecturette or some other information that is to be provided by 
you. Consequently, with just a little homework, you should be in a position 
to display the transparency and expand the abbreviated points through a 
lecturette or directions. The key is to free yourself from the podium or 
central stage and walk around while delivering the message. 

Warning; Do not attempt to deliver this program without first reviewing and 
coordinating the transparencies with the various activities in Section 3 of 
the workshop. 

Do not use the program in a structured lecture hall where participant 
interaction is restricted. The exercises and program are designed for 
interaction, not a lecture. 

Equipment and Materials Needed 

o Newsprint paper and markers 

o Masking tape 

o Workbooks 

o Overhead projector 

o Spare projector bulb 

o Blackboard 

o Transparency markers 
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SECTION 3: PRESENTING THE WORKSHOP 
PROGRAM EVALUATION 



TIME 



MATERIALS 



Total 
Time; 
15 min. 



5 mim 



\ 



Activity 1: Introduction and Objectives 



Leader Focus : The purpose of Part 1 is to provide 
the participants with easy, highly interactive 
experiences in order to move towards an open, free 
and trusting workshop. Keep this section fast 
moving. Attempt to honor and confirm the worth of 
school boards. This is an inclusion exercise. 

o Leader Instructions : Distribute the participant 
booklet, "A Work Session." 

o Review the workshop objectives. Either write 
them on newsprint for everyone to see or use 
the transparency provided. At the end of the 
workshop, the participants should identify 
the following: 

— Program evaluation and how it is used 

— The school board and program evaluation 

— How program evaluation can help school 
board members 

— Some of the questions program evaluation 
\ can answer 

\ 

— Some of the questions school board members 
need to ask about program evaluation 

o Instruct the participants to turn to the 
introduction Exercise 1, "Where Are You?" 
Instruct the participants to take about five 
minutes to answer the questions. 

o Next, if the workshop is organized in small 
groups, instruct the group members to take 
about five minutes to share their answers in 
their groups. If there are.no small groups, 
have the participants share with a neighbor. 



Newsprint or 
Trans. PE 1 
Participant 
Booklet 
Page 1 



Participant 
Booklet 
Page 2 
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TIME 



MATERIALS 



10 rain. o After the, participants have had a chance to - Trans. PE 2 

answer and discuss the questions on "Where Are 
You," ask for a show pf hands to the following 
questions. (Count the audience reaction and 
write the count on newsprint or a transparency.) 

— 'HOW MANY SAID THE BOARD SPENDS LESS THAN 

10 PERCENT OF ITS TIME ON CURRICULUM 
MATTERS? 

— HOW MANY SAID THE BOARD SPENDS BETWEEN 10 * 
AND 20 PERCENT OP ITS TIME ON CURRICULUM 
MATTERS? 

— HOW MANY SAID THE BOARD SPENDS BETWEEN 20 
AND 30 PERCENT OF ITS TIME ON CURRICULUM 
MATTERS? 

— HOW MANY SAID THE BOARD SPENDS BETWEEN 30 
AND 40 PERCENT OF ITS TIME CN CURRICULUM 
MATTERS? 

— HOW MANY SAID THE BOARD SPENDS BETWEEN 40 
AND 50 PERCENT OF ITS TIME ON CURRICULUM 
MATTERS? 

— HOW MANY SAID THE BOARD SPENDS BETWEEN 50 
AND 60 PERCENT OF ITS TIME CN CURRICULUM 
MATTERS? 

— HOW MANY SAID THE BOARD SPENDS BETWEEN 60 
AND 100 PERCENT OF ITS TIME CN CURRICULUM 
MATTERS? 

o Allow time for participant reaction and go onto 
the next item. Record the answers on 
Transparency PE 2. 

o Continue on and ask for a show of hands to Trans. PE 3 

the following question: 

HOW MANY OF YOU HAD A LARGER PERCENTAGE 
NUMBER FOR QUESTION #2 THAN #1? 

o Ask, HOW MANY OF YOU 'WOULD LIKE TO SPEND... . 
(Make a count and write it- down.) 

— 0-25 PERCENT OF YOUR BOARD TIME ON THE 
LEARNING PROGRAM 
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— 25-50 PERCENT OP YOUR BOARD TIME ON THE 
LEARNING PROGRAM 

— 50-75 PERCENT OF YOUR BOARD TIME ON THE 
IEAENING PROGRAM 

— 75 PERCENT OR MORE OF YOUR BOARD TIME ON 
3HE LEARNING PROGRAM 

Share and review the count of what they would 
like to do. 

Ask them, IF THE SCHOOL BOARD IS IN CHARGE , WHY 
DON'T YOU DO IT? 

Lecturette; A 1979 NSBA survey of new school 
board members indicated they expected 
curriculum to be the biggest issue with which 
they would work. 

New board members expected curriculum to take 
about 27 percent of their time. 

Boards actually spent about 6 percent of their 
time on currxculura related matters. 

The number one reason board members resign or 
refuse to serve again is due to their lack of 
involvement in the learning program. 

Emphasize the point that the school board, 
through policy, can direct the superintendent 
to spend 20 , 30 , 40 percent or more time of 
each board meeting on curriculum an<3 school 
improvement. 

Emphasize the point that if the board wants to 
work on school improvement, it must provide for 
program evaluation. 



15 



TIME 



MATERIALS 



Total 
Time: 
20 rain. 



5 rain. 



5 rain. 



Activity 2: Defining the Learning Program 



Leader Focus : The purpose of this section is to 
define the elements of the learning program — 
planned experiences to achieve pre-planned goals. 
This activity is designed to be an inductive 
experience and to facilitate group interaction. 

o Leader Instructions ; Have each group (table) 
develop a list that represents in their view 
the three or four most important goals for the 
schools. Tell thera they have 5 minutes. 

o Have each group share their list within their 
group. 

o Lecturette : In a 1981 publication from 
University of California at Los Angeles, 
Dr. John Goodlad reported that all education 
goals can be combined into four areas* 

o The list shoula include: ' 

— Academic goals 

— Social goals 

— Vocational goals 

— Personal development goals 

o Leader Instructions ; Have a few of the tables 
share their list with the total group. 

o Next ask the participants to develop a list of 
the items the "board pays for" to help the 
students reach the goals. 

o Give the participants time to develop their own 
lists before you share the transparency. 

o Ask the groups to give reports. 

The listlshould include: 

— Textpooks 

— Teachers 

— Principals 



Trans. PE 4 



Trans. PE 5 
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Superintendents 

— Teacher aides 
Custodial staff 

— Counselors 

— Others in the school 

— Learning materials 
Physical facilities 

— Etc, 

Next ask the participants to list the things 
teachers do to help students achieve goals. 

— Use Edgar Dale's Cone of Experiences 
to bring this together after the 
participants have shared their ideas. 

Now define the "learning program" (curriculum) 
as the actual experiences provided pupils by 
the school to realize goals. 

Summarize and tie in goals and experiences. 

Lecturette s Discuss other experiences students 
get over which the board has no control: 

— The students themselves (their abilities, 
motivation) 

— . Community expectations 

— Parent expectations 

List or discuss indirect experiences that 
influence learning: 

— The teachers and th£-~wajr they teach (both 
their professional expertise and their 
human side — how they work with people) 



The administrators and' the way they "run 
the schools" (both their professional 
expertise and administrative skills and 
their humanness in working with teachers, 
a ides / students, and all employees) 

The superintendent, who works, with the 
district and diroctly with the board 
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— You, as a board member and the decisions 
you maker the values you hold, the goals 
you set for the district, and the amount 
of time you spend at board meetings on the 
curriculum 

Define the learning program again. 

Continue on by saying: 

TODAY WE ARE TALKING ABOUT THE EXPERIENCES THE 
STUDENTS IN YOUR DISTRICT HAVE IN THEIR 
SCHOOLS. YCO, AS BOARD MEMBERS , WANT TO MAKE 
SURE THOSE EXPERIENCES ARE WORTHWHILE AND LEAD 
TO ACCOMPLISHING THE GOALS YOU HAVE SET FOR 
YOUR DISTRICT* 

Stop and ask for questions or comments. 
Summarize, 



Trans, PE 7 



Helpful Hint ; Try to move this section very 
quickly; however, it is important to allow 
for group inclusion. It is also critical 
that the participants have some idea as to 
those "things" over which they have control 
that affect learning. 
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Total 
Time: 
25 rain. 



5 min. 



5 min. 



Activity 3; Evaluation Exercise 



Leader Focus ; The purpose of this activity is to 
provide a set of common experiences that will move 
the participants toward a two-step definition of 
evaluation: INFORMATION-JUDGMENT. The purpose here 
is to demonstrate that we all make judgments and 
evaluations. The problem is, how good are they? 

o Leader Instructions : Now ask the participants 
to turn to Exercise 2, Program Evaluation, and 
complete the five questions on the worksheet. 
Tell them, YOU ARE TO THINK OF YOUR OWN 
DISTRICT WHEN YOU ANSWER THE QUESTIONS. YOU 
WILL HAVE FIVE. MINUTES TO COMPLETE THE EXERCISE 
AND TO SHARE IT WITH YOUR GROUP. 

o The object is not to have participants share 
their answers, but to have them provide an 
answer to these questions. In answering the 
question, the pe'rson "collects or identifies 
information then judges that information to 
make a decision (which is the definition for 
program evaluation). That is, "Evaluation is a 
process of gathering information and making 
judgments." 

o Ask the participants to respond by a show of 
hands : 



Participant 
Booklet 
Page 3 



DID YOU IDENTIFY THE TEACHER WITH THE MOST 
EFFECTIVE READING PROGRAM? RAISE YOUR 
HANDS IF YOU DID. 

WOULD. YOU VOTE TO MAINTAIN YOUR DISTRICT'S 
MATH PROGRAM AS IT IS NOW? RAISE YOUR 
HANDS IF YOU SAID YES. RAISE YOUR HANDS 
IF YOU SAID NO. 

DO STUDENTS GRADUATING FROM VOCATIONAL 
COURSES HAVE THE SKILLS NEEDED FOR THAT 
VOCATION? RAISE YOUR HANDS IF YOU SAID 
YES. 

WHO IS THE "BEST TEACHER IN THE DISTRICT? 
HOW MANY IDENTIFIED THE BEST TEACHER? 
RAISE YOUR HANDS P 
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— SHOULD STUDENTS BE REQUIRED TO TAKE 
CHEMISTRY BEFORE GRADUATING FROM HIGH 
SCHOOL? HOW MANY SAID YES? RAISE YOUR 
HANDS IF YOU DID* 

i 

o Make the point that the participants are Trans. PE 10 

evaluators because they just made a whole 
series of judgments. The problem may be: 

~ DID YOU HAVE GOOD INFORMATION? 

WHERE DID YOU GET YOUR INFORMATION? 

10 rain. o Lecture tte : Point out that each of these 

questions involves some sort of evaluation, 
valuing or judging to get an answer. That 
answer may depend on the information on which 
the judgment or evaluation is made. 

o Comment on the types of information . Tell them Trans. PE 11 
that they are all good, but we all need a 
number of sources on which to make a judgment: 

— Intuition 

Personal likes/dislikes 

— Opinion of neighborhood 

— Test data 

— Test data collected over 10 years 

— External evaluation team 

Winning state championship two years out 
of last three 

Adoption of new reading ptogram, 
comparison evaluation of that program with 
program existing in other schools in 
district 

Reading a research report of math program 
used in your district's elementary schools 

— Report by reading specialist at last 
month's board meeting 

Advisory committee study of high school 
science programs 
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o There are many, many criteria on which to base 
a judgment* 

o A single criterion may be incomplete or an 

inadequate basis for a board's decision and you 
want to identify the basis on which your board 
makes decisions. 

— For example: A new neighbor calls to 

complain that the elementary math program 
provided to her daughter is not as good as 
the one in their previous location 
(Wherever 9 USA) • 

Possibilities: 

The math program really needs some 

revision and updating, 

That particular neighbor has a 

personal dislike of the teacher. 

The student did not do homework for 

math class, got a zero for thd daily 
class work and came home and 
complained about math. The 
parent/neighbor took the daughter's 
word without investigation. 

The teacher in the program is a year 

from retirement and has not kept up 
to date regarding teaching 
mathematics. 

— The district's math inservice program 
is not sufficient. 

o On and on and on. Moral of the story: One 
complaint is not a sufficient basis for 
changing the math curriculum. 

o You do not know when you hear a complaint from Trans. PE 12 
a neighbor or patron if that complaint is an 
isolated comment or the tip of an iceberg, 
indicating that there are problems f until you 
have more information. 
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Any of these possibilities might be true. 
Without more investigation or more information, 
you simply do not know the extent of the 
problem. 



5 min. o 



Our definition of evaluation is that EVALUATION 
IS A TWO-STEP PROCESS . THE FIRST STEP IS 
GATHERING INFORMATION . THE SECOND STEP 
INVOLVES JUDGING THAT INFORMATION TO MAKE A 
DECISION , (Read the transparency.) 

If you make a decision based on one parent : s 
complaint, then you are set up for reactions to 
other complaints. 

If you need to make a decision for political 
reasons, fine. Do so — but be willing to define 
Jiose reasons. 

You want to be in control while being 
responsive to your community, but not to be 
"reactive" to every complaint or to overreact 
until you know the full story. You don't want 
to start a full scale evaluation on the basis 
of one complaint. 

Allow time for questions and reactions. 

Summarize by saying that, for the purposes of 
this workshop, evaluation is defined as a 
two-step process; 



Trans. PE 13 



Trans. PE 13 



1. Identify and collect "information 

2. Judge and place value on that information 
so that a decision can be made 



Reusing Iceberg transparency, continue suntmary 
by saying, WHEN YOU HEAR A COMPLAINT OR GET A 
QUESTION f YOU DON'T KNOW IF YOU HAVE AN ICE 
CUBE OR THE TIP OF AN ICEBERG UNLESS YOU HAVE A 
PLANNED SYSTEM OF EVALUATION. 
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Total 
Times 
25 rain. 



15 rain. 



5 rain. 



Activity 4: what is the Board's Role? 



Leader Focus : The purpose 6f this section is to 
establish the notion that the board's specific 
responsibility is to guarantee a quality learning 
program within the requirements of state law and the 
needs of the local school district. 

o Lecture tte ; Ask and answer the question, WHAT ■ 
IS THE BOARD'S ROLE IN THE LEARNING PROGRAM? 

o THE SCHOOL BOARD HAS A RESPONSIBILITY FOR 

PROVIDING FOR A PROGRAM OF QUALITY INSTRUCTION 
WITHIN AVAILABLE RESOURCES (NWREL Keys project, 
Four Functions paper). 

o LOOK AT THOSE WORDS, "QUALITY" PROGRAM, 

"AVAILABLE RESOURCES." (Transparency PE 14). 
THOSE WORDS IMPLY JUDGMENT, DECISIONS. HOW DO 
YOU KNOW IF THE CURRICULUM IS SOUND? 

o Leader Instructions ; Next, tell the partici- 
pants to turn to Exercise 3, page 4, "Who Is 
Responsible." Read the instructions and 
ask them o> respond to the statements. After 
they complete the exercise as individuals, have 
them share their responses with their neighbors 
or their "small group. Tell them they have 15 
minutes to complete Exercise 3. 

o Lecturette ; After they have completed the 

exercise and have had time for discussion, you 
might point out that: 

— There are no clear-cut right/wrong answers 
that apply in all districts. 

Answers should vary according to district. 

— Answers will vary according to operating 
style of the school board and the 
superintendent. 

The important point to be made is that 
each district should decide the answers 
for itself. 

— The superintendent and the board for each 
district must decide for themselves. 



Trans. PE 14 
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o If the board sets up goals and philosophies of 
the district, hires a professional staff , 
allocates monies to purchase textbooks , adopts 
curriculum plans, and so on, has it fulfilled 
its responsibility? 

o Nol It has a responsibility which is ongoing. Trans. PR 15 
and continuous* It must ask questions and seek 
information about the learning program. It 
must insure that the learning program is 
evaluated . 

1. Is the curriculum sound? 

2. Is the instructional staff implementing 
the plan? 

3. Are the students learning what was desired 
and planned for them to learn? 

i 

o The board's role is to insure quality programs 
through setting a policy for a systematic plan 
for program evaluation . 

o The board should provide for program evaluation 
by its policies. 

o An ongoing, systematic plan for evaluation of 
curriculum which fits with the size of your 
district and is budgeted and planned in advance 
is better than a response to a "crisis." 

o You do not have adequate information about your 
school programs until you have a planned, 
systematic way of evaluating the learning 
program. 

o Point out that intuition is okay. It is a 
valid part of information, but it may not be 
sufficient basis on which to adopt, discontinue 
or revise a curriculum. 
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5 min* o 



o 
o 



Point out it is the boards responsibility 
to see to it that a curriculum plan is 
developed , implemented and evaluated. 

Is the curriculum meeting the needs of the 
students and the school district? 

Here's what the evaluation cycle looks like. 

The development curriculum circle is a dotted 
line because once it is developed , the cycle 
becomes a continuous process. of looking at that 
curriculum and continuing it or revising it as 
needed • 



Trans. PE 16 



Helpful Hint ; Participants may say, "We 
don't have time to do all the things we are 
already doing or that we need to do now." 
"Why can't we leave those things to* the 
professional educators that we have hired?" 
Emphasize that the board won't usually be 
involved in doing the evaluations but should 
provide for the evaluations to be done, and 
the board should insure that the information 
from evaluation is useful and timely . In 
small school districts the school board may 
need to get help from an outside agency, 
preferably the state school boards 
association. Emphasize Transparency PE 16. 
Drive home the point — a "planned system." 
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Time: 
5 min, 



Activity 5: The Board's Role 



Leader Focus : The purpose of this exercise is to 
establish a list that starts to define the school 
board's role in program evaluation. The list will 
never be complete and will vary from district to 
district. 



Leader Instructions ; Tell the participants to 
turn to Exercise 4, page 5 of their workbook. 



Lecturette : Use Transparency PE 17 and review 
the- list. 

Ask the participants if they can add to the 
list or if they have any comments. 

Go back over Transparency PE 16. 



Participant^ 
Booklet 
Page 5 

Trans. PE 17 



Trans. PE 16 



Helpful Hint : Drive home the points listed 
on Transparency PE 17. 
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Total 
Time: 
15 rain. 



Activity 6; Use of Evaluation Information 



Leader Focus : The purpose of this exercise is to 
move the participants from the theore-ical notions 
of evaluation information to the uses of evaluation 
information. 



10 rnin. 



rain. 



Leader Instructions : Have the participants turn 
to page 6 of their workbook, Exercise 5. 

Tell the groups they have 10 minutes to develop 
a list of ten ways on how a school board would 
use information from evaluation. 

Remind them that each group will give a report 
at the end of the time. 

After each group has reported, use 
Transparency PE 18 to review some of the uses 
of evaluation results. Tie in their ideas with 
the transparency. 



Participant 
Booklet 
Page 6 



Trans. PE 18 
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Total Activity 7: Review and Legal Basis 
Time: 
10 rain. 

Leader Focus ; At this point, you should review 
where the participants have been in the workshop. 

5 min. o Lecturette: First, we established that school 
board members wanted to become more involved in 
the learning program. * 1 

o Second, we established that they could become \ 
more involved by directing the superintendent \ 
to spend more of the board's time on curriculum \ 
evaluation. - 

o Third, we defined the learning program as the 
"actual experiences provided by the school to 
realize goals." 

o Fourth, we defined evaluation as a two-step 

process: collecting information and making i 
j udgments . \ 

o Fifth, we explored the school board's role in \ 
program evaluation. \ 

\ 

o At this point, you should summarize and 
establish the board's responsibility and 
authority for the school program and i^3 j 
evaluation: 

1. Quote state laws or regulations 
appropriate for that state." (Look them up 

for your state.) x 

2. State project philosophy or quote NSBA or 
other authorities* (See quote below.) 

3. Community expectations. 

o Your State Law (Quote it here.) 
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5 rain. o Quote Harold Webb, NSBA Former Director, 1976: 

"Boards have the responsibility to ask three Trans. PE 15 

questions: 

1. IS THE CURRICULUM SOUND? 

2. IS THE INSTRUCTIONAL STAFF IMPLEMENTING 
THE PLAN? 

3. ARE THE STUDENTS LEARNING WHAT WAS DESIRED ' 
AND PLANNED FOR THEM TO LEARN?" 

o Quo£e Thomas A. Shannon, Executive Director, Trans. PE 19 

National School Boards Association: * 

"SCHOOL BOARD MEMBERS ARE ULTIMATELY 
RESPONSIBLE AND ACCOUNTABLE FOR THE QUALITY OF 
SCHOOLING. THIS RESPONSIBILITY EXTENDS 
SPECIFICALLY TO PROVIDING FOR EVALUATION OF 
TEACHERS, INSTRUCTIONAL MATERIALS, PRINCIPALS 
AND SUPERINTENDENTS AS WELL AS THE OVERALL 
ENVIRONMENT FOR 1 LEARNING ACHIEVEMENT . 1 " 
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Total 
T me: 
40 min. 



5 min. 



Activity 8: Questions 



Leader Focus : The purpose of this section is to 
apply the concept of evaluation • 

o Leader Lecturette : Exercise 6 is constructed 
around ■ seven concerns : 



20 min. 



— Reading 

— Math 

Foreign Language 

— Social Studies 
Bilingual Education 

— Sex Education 

— Talented and Gifted 

Instructions : Each group should choose or 
be assigned one curriculum concern* The 
curriculum concerns are on pages 12-18 of their 
workbook. 

Do not let all the groups work on the same 
curriculum concern. 

Give each group three pieces of newsprint paper 
and one marking pea. 

Tell the groups they are to put their answers 
on the newsprint for a report to the group, and 
they should pick someone to report — NOT A 
SUPERINTENDENT . 

Allow 15-20 minutes to complete the exercise. 

Have the participants turn to page 7, Exercise 6 
of theix workbook. Review the instructions. 
They are to: 

— Read the assigned curriculum concern on 
pages 12-18. 

— Determine what decision needs to be made. 



Participant 
Booklet 
Pages 12-18 



Newsprint 
Felt Pen 



Participant 
Booklet 
Page 7 



— Develop a list of questions that need to 
be asked. 
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— Select the three most important questions* 

List three pieces of evaluation 
information they would like from the 
superintendent. 



15 rain. o 



— Place their group answers on the newsprint, 

At the end of the Program Analysis exercise , 
provide time for reports to the large group. 
Note: Move from group to group, calling on 
different groups to respond to either parts 1, 
2, 3 or 4 on the Program Analysis worksheet. 
Then summarize and move on to the next exercise. 
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10 rain. Activity 9; Summary 



Leader Focus ; The purpose of the summary activity 

is to draw the workshop to a close. 

o .Use Transparency PE 1 to review the workshop 
objectives. 



Trans. PE 1 



Relate the exercises in the workbook to the 
objectives. 

Objectives of the workshop were to stimulate 
discussion on program evaluation to identify: 

1. Program evaluation and how is it 
used — Exercise 2, Program Evaluation* 

2. The school board* s role in program 
evaluation — Exercise 3, Who is Responsible 
and Exercise 4, Board's Role. 

3. How program evaluation can help school - 
board members — Exercise 5, Use of 
Evaluation. 

4. Some of the questions program evaluation 
can answer — Exercise 6, Program Analysis. 

5» Some of the questions school board members 
need to ask about program evaluation — 
Exercise 6 and Exercise 7a and 7b. 



Lecturette ; Ask the participants to turn to 
pages 8 and 9 in their workbook. 



Participant 
Booklet 
Pages 8-9 



At this point you might say, THERE ARE SOME 
ADDITIONAL ISSUES A BOARD NEEDS TO THINK ABOUT 
IN REGARD TO EVALUATION. 

Then go over and comment on each of the items 
on pages 8 and 9. 

This may be a good place for your association 
to "sell" some of your services. 

Review page 10 of the participant book. Use 
Transparency 20 to guide your review* 



Trans. PE 20 
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10 min. Activity 10; Evaluation of Workshop 

Leader Focus : Let participants know you are 
modeling behavior. "I'm having you evaluate this 
session, just like you should evaluate yourselves as 
a school board , just like you yourselves should 
evaluate your people and programs. If you don't 
evaluate, how else can you grow or improve? 

o Leader Instructions ; Hand out the evaluation Evaluation 
forms or refer the participants to pages 22 Form 
and 23 in the workbook. Pages 22-23 

o Ask for individual evaluations. Tell them they 
have 10 minutes. 

o Collect/ read and tabulate the results. 
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SECTION 4: EVALUATING THE WORKSHOP 



Evaluation is the hallmark of a professional — a good teacher or workshop 
leader. A systematic planned evaluation is the only method we have to 
determine the effectiveness of our instruction and the achievement of the 
learners. This program was developed around four techniques of development 
and evaluation. 

Topic teams and state review committees identified the goals of the workshop, 
and in souve cases, the techniques. In addition, pilot tests, field tests and 
expert reviews were used at appropriate times during the development and 
evaluation phases. 



*A summary repprt of evaluative procedures and results is on file in the 
Rural Education Program of the Northwest Regional Educational Laboratory, 
300 S. W. Sixth Avenue, Portland, Oregon 97204. 
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SUMMARY REVIEW OF DEVELOPMENT AND PILOT TEST ACTIVITIES 



PROGRAM EVALUATION 

Topic Team 

The Oregon component of the school board inservice program was responsible for 
the development of three topics: (1) Policy Making, (2) Communicating with 
the Community, and (3) Evaluation. The Executive Director of the Oregon 
School Boards Association (OSBA), Mr. Tom Rigby, appointed one OSBA staff 
member to work with each topic: Mrs. p. Fitzwater, Policy; Mr. J. Marten, 
Evaluation; and Mr. D. Dixon, Communicating with the Community. Mr. Rigby 
also appointed representative school board members and superintendents to 
serve on each topic committee. 

State Review Committee 

In addition to the topic teams made up of school board members and 
superintendents, Mr. Rigby appointed and convened a State Review Committee. 
The purpose of the State Review Committee was to provide a forum for the 
involvement of opinion leaders from allied educational agencies. In 
attendance at the meeting on January 24, 1979 were: 

Mr. Tom Rigby/ Executive Director, OSBA 

Mrs% Pat Fitzwater, Training and Policy Specialist, OSBA 

Dr. Milt Baum, Oregon Department of Education 

Dr. Wright Ccwger, Willamette University, Professor and school board member 
Dr. Ron Petrie, Dean, School of Education, Portland State University 
Dr. Jens Robinson, Superintendent, Woodburn, Oregon 

Dr. Ray Mullen, Superintendent, Tillamook Educational Service District 
Mrs. Carol Williams, board member and President of the OSBA 
•Dr. Betty Tomblin, Northwest Regional Educational Laboratory 
Dr. Leslie Wolfe, Northwest Regional Educational Laboratory 

The State Review Committee made five recommendations: 

1. The inservice programs should help clarify the role of the school 
board in public education. 

2. She inservice programs need to emphasize the proactive role that can 
be played by an informed school board member. 

3. Ihe inservice programs must emphasize the dynamics that take place 
between board members, between the board and the superintendent, and 
between the community and the board. 

4. The inservice programs must also be targeted to superintendents. The 
inservice materials should help create a dialogue between 
superintendents and school board members relative to their working 
relationship in a democracy. 
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5. The inservice program should be developed on several levels which 
should include materials that a superintendent can use with 
prospective or existing school board members. 

Itopic Team Explorations. 

Program Evaluation topic team had four meetings between February 1979 and 
September 1979. The topic team suggested a workshop that would place emphasis 
on the ways in which program evaluation information can be used to help board 
members, such as in specifying the basis of decisions, communicating with the 
community and becoming more knowledgeable about their district program. 

The roles of the board member in program evaluation stressed in the workshop 
were to be: 

o Ask questions— description of program and impact of program 
o Set policy to insure ongoing evaluation within district 

o Use, evaluation information to help them make decisions about programs 
aid policy 

Developmental Review 

The first meeting discussed the goals and objectives and a general outline for 
the workshop. The last meeting of the topic team reviewed the first version 
of the workshop. In addition, the workshop was also tested and reviewed with 
two different groups of school board members and superintendents at the 
Northwest Regional Educational Laboratory. 

Pilot Test Activities 





Place 


.Date 


No. of 
Participants 


Presenter 


1. 


Anchorage, Alaska 


Nov. 


1979 


58 


Tomblin 


2. 


Boise, Idaho 


Nov. 


1979 


76 


Tomblin 


3. 


Helena, Montana 


Nov. 


1979 


45 


Tomblin 


4. 


Portland, Oregon 


Nov. 


1979 


52 


Wolfe 


5. 


Spokane, Washington 


Dec. 


1979 


232 


Tomblin 
& Wolfe 
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Summary of Pilot Test Activities 



Evaluation Methodology 

The package in its original pilot test form was tested in 4 states with a 
total of 600 people, of whom 93 percent were school board members. The pilot 
tests employed a "pretest-treatment-posttea t" format. The participants were 
informed that the materials were in a pilot test form and that their 
cooperation and their evaluations would be roost helpful. All of the 
participants agreed to complete pre-workshop and post-workshop questionnaires. 

The results of pre- and post-workshop evaluations for all participants are 
presented in Table 1. Table 2 presents the data from only the school board 
members. The responses to the open-ended statement are not presented in this 
document, but they were used in adjusting certain parts of the pilot test 
workshop. 

Evaluation 

There have been two major reviews of this workshop, one by the project staff 
(internal) and one by the topic team (external). The review by the topic team 
resembled a pilot test in that the topic team were participants. After the 
workshop was presented, they expressed their reactions on a written critique 
and then reviewed the materials in a discussion. In ^addition, there were two 
pilot tests of the entire workshop, each test being conducted with school 
board members and superintendents ♦ At the conclusion of each review or pilot 
test, information was collected from the participants on a standard workshop 
critique form (See Table 1). 

A major purpose of the pilot test in this instance was to determine if- the 
language, the ideas, the "content/ and the level of the approach to program 
evaluation was appropriate to school board members. Program evaluation is an 
area in which few board members or few school boards are heavily involved; 
therefore, this workshop endeavored to find the entry level to introduce 
school board members to their role in program evaluation, it was also 
intended to be a model on which further information and workshops, materials 
and products could be developed to give board members more in-depth 
information about involvement in program evaluation. 

Both reviews and pilot tests determined that the ideas and content presented 
in the workshop were indeed relevant and useful to school board members. They 
also indicated that the material is appropriate for all school board members 
and is not restricted to newly elected board members nor is it restricted to 
veteran board members or board members in a certain locale (See Tables 2 
and 3) . 

A summary of the pilot test evaluations from more than 600 school board 
members and superintendents who have participated in the Program Evaluation 
workshop confirms the value of the workshop to the participants. In response 
to the evaluation question, "Did the content help you as a school board 
member?" the average response was 4.47 on a 5-point scale. One caution should 
be noted. The workshop should not be pushed into any time frame less than 90 
minutes. Smaller time frames do not allow * iequate time for appropriate group 
discussion, which results in some level of dissatisfaction among the 
participants . 
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Typical Comments 



Examples of comments made by participants when asked to state one thing they 
learned from the session included these representative phrases: 

o The importance of clarifying issues prior to substantive discussion 

o Useful problem solving model and procedures 

o Helpful hints on prioritization 

o That I need further education in evaluation 

o That the Lab does make use of feedback from pilot groups 

o Supported by interest in increasing effort to affect curriculum 
evaluation 

Expert Review 

The third and final review of the material was conducted on two levels, one by 
the National School Boards Association and a second and independent evaluation 
by Dr. Keith Goldhammer of Michigan State University. The materials and 
procedures at this level were reviewed and evaluated for the following: 

1. Hieir representation of the existing basis of knowledge in the 
literature c\nd the profession 

2. The procedures and methods outlined in the workshop and leader 1 s guide 
Changes 

A number of changes evolved as the workshop was evaluated and reviewed. The 
most dramatic change was the expansion of the "visuals" to the "lecture" part 
of the workshop and the addition of the "questionnaires" that can be used by 
the board members. 

Field Tests 

Field tests were defined as those occasions when the workshop was delivered by 
a person other than a member of the NWREL staff. The workshop was delivered 
in five states by six different association staff people to more than 900 
school board members and superintendents. 

The recommended changes from the pilot tests, field tests c .id expert reviews 
have been incorporated in this package. 
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Program Evaluation Workshop 
Summary of Pilot Test Data 




Questions: 








f 


OKI Pr "f vpr Plpfli*? 

vUJ CutlVCD \jlCCt JL • 


Yes - 50% 




Yes - 100% 


91% 


Objectives Met? 


Yes - 50% 




Yes - 


91% ' 


Ideas - Relevant to Board Members 


Yes - 75 % 


d 
o 

♦H 


* 

Yes - 100% 


45% 






CO 
CO 




NA 27% 






3 






Willingness to Discuss 




CO 

•H 






Program Evaluation 


N/A 


Q 


More willing 


More willing 




r-* 

rt 


67% 


-82% 






Xi 






Overall Reaction to Workshop 


Minor Change 


U 
0) 

> 


As is - 50% 


As is - 64% 


75% 




Minor Change 


Minor Change 








50% 


27% 


Appropriate For All Board Members 


N/A 




Yes - 100% 


82% 


Number of Participants .Total 


5 


3 


7 


11 


Board Members 


3 




4 


6 


Superintendents 


1 




2 


4 


Other 


1 (no data) 


3 Staff 


1 (no data) 


1 
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Table 2 



The Board 1 s Role in Program Evaluation 
Pilot Tests 

State Convention Workshops - November-December, 1979 




Amount of time for session: 


lH hours 


3h hours 


2 hours 


l*i hour 8 


1 hour 


Questions: 


















Did workshop cover what was 
advertised? 


3.50 


4.21 


N/A 


N/A 


N/A 


N/A 


N/A 


N/A 


Content of help to you' as SB member? 


A. 00 


4.47 


4.33 


5.00 


4.49 


4.02 


4.21 


4.33 


Specific actions to take back re: 
program and /or its evaluation? 


3.73 


3.95 


3.50 


4.07 


4.44 


4.00 


3.85 


4.07 


Adequate time for questions? 


2.77 


4.11 






2.49 


3.88 


2.63 


3.07 


Overall rating of workshop: 


3.29 


3.50 








3.87 


4.04 


3.93 


Scale: 1 = low, 5 = high 


















*Threse ratings were mixed with other 
sessions presented throughout the 
day and do not reflect reaction to 
Program Evaluation workshop only. 
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Table 3 

Summary of Evaluation Workshop Results From Five State Conventions 



Question 


Mean* 


% 


i. 


Do you feel that the content of this workshop will be 
of help to you as a school board member? 


4.49 




2. 


Is there one (or more) specific action(s) you can take 
back to your district re: program and/or evaluation? 

Percentage naming one: 


4.44 


53.3% • 


3. 


Was adequate time allowed for questions and/or discussion 
"during the presentation? 


3.80 




4. 


Percentage stating things learned or gained from sessions 




82% 


5. 


Overall workshop rating: 






* 


Mean is based on scale of 1 B low, 5 ■ high 

• 

• 


4.30 
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THE SCHOOL BOARD'S ROLE IN PROGRAM EVALUATION 
Workshop Critique 

1 

In order to improve the workshop, we would appreciate your time in answering 
the following questions. Peel free to make comments regarding any item. 

1. Did this workshop cover what it was advertised to cover? 

No Somewhat Yes 
I ? 3 4 5_ 

If not, what was missing that you expected to cover? 



2. Do you feel that the content of this workshop will be of help to you as a 
school board member? 

No Somewhat Yes 
1 2 3 ' 4 5 



3. Is there one (or more) specific action (s) you can take back to your 
district concerning program and/or its evaluation? 

Definitely Definitely 
Not . Somewhat Yes 
1 2 3 4 5_ 

Can you name one? 



4. Was adequate time allowed for questions and discussion during the 
presentation? 

No Somewhat Yes 
_i 2 3 4 5 



5. Please state one thing you learned or gained from the session: 
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6* What questions remain unanswered for you regarding program evaluation? 



7. Relative to workshops you have attended, how do you rate this workshop? 

Very Poor Very Good 

1 2 43 4 5 



8. How would you rate the overall performance of the leader— knowledge and 
presentation? 

Very Poor Very Good 



1 



5 



9. Are you: 



A school board member » 
A superintendent 
Other: please state: 



10. What can the leader do to improve the session? 



11. Any additional comments for improving this and future workshops will be 
appreciated* 
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EVALUATING INSTRUCTIONAL PROGRAMS* 



"the governing board of every school district 
shall, evaluate its educational program, and shall 
^ make such revisions as it deems necessary 

California Education Code 

Every time you irespond to a question about your curriculum (such as, "Do 
you have a good reading program in your district?"; "How are our students 
doing in math?"; "Is our elementary science program adequate?") you are 
making an evaluation, and you go through two steps. First, you collect or 
identify information about the program. Second, you judge or place a value on 
the worth of that program. 

Therefore, you need to consider your sources of information. Do you have 

data collected in some systematic way from the appropriate group in addition 

to the informal sources that every board member has? There are times when an 

angry parent or community member wishes the board to take action; the old 

saying about the squeaking wheel getting the attention is sometimes true and 

we may react before we find out if that one concerned per yon is expressing a 

comment representative of others or if it is an isolated incident. Do we have 

t 

an ice cube or the tip of an iceberg? Is it a small incident that can be 
solved between the persons involved or by providing information or is it 
indicative of a real problem ^thin the curriculum or instructional program? 



*Torablin, B. T., & Wolfe, L, G* Evaluating Instructional Programs. 
Sacramento, CA: California School Boards Association Journal, September 1980. 
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Ongoing systematic evaluation can help you determine which is*the case* It 
can help you be leaders and take a role of positive action rather than a 
reactive or crisis-centered role. 

What we want to do in this article is to emphasize the importance and uses 
of program evaluation for school boards and provide you with a process for 
beginning a systematic cycle of program evaluation within^your district. We 
will present some specific ideas of the board's role in evaluation and of the 
payoff to you .and your community, and especially to your students for whom th- 
schools and the curriculum were established. We cannot overemphasize the 
value of approaching program evaluation vith a very positive attitude and 
concern for improving the program. This will combat the threatening aspects 
of evaluation, particularly to those who are unfamiliar with evaluation or who 
may not understand your intention. 



Program evaluation, as discussed here, is a systematic collection of 
information about a curricular program which includes a judgment of worth and 
recommendations for improvement of and decisions about the program. The 



course, decisions do have to be made to cut back or discontinue programs. 
Even then, program evaluation can help insure the decision is based on good 
# information rather than opinion. Evaluation is an important part of a well 
functioning program cycle which is an ongoing process to continually improve 
the curricular program. Phases of that cycle are: ^1) planning, 
(2) implementation, (3) evaluation, and (4) revision, if necessary? then the 
cycle bfKjins again. 




What is Evaluation? 



primary purpose of program evaluation is not to prove but to improve . Of 
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PLANNING 



REVISION" 
(IMPROVEMENT) 



IMPLEMENTATION 



EVALUATION 



Figure 1. Curricular Program Cycle 



Revise , 
Act on 


f Improve, 
Decisions 






r 






Use Evaluation 
Information for 
Decisions 









Measure 
Impact 



Goals & Philosophy 
of District 



Planning £ Development 
of Program 



Implement Program 



Evaluation 



Board Policy 
for Evaluation 



Describe 
Program 



Figure 2. Board Involvement in Program Development and Evaluation 
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How Can It Help You? 
Evaluation of learning programs is a very important vehicle for school 
board involvement in the educational program and the payoff for boards and 
students (as well as teachers and administrators) can be tremendous. In 
addition to helping you make decisions about programs (such as whether to 
adopt, expand, cut back, etc*) evaluation helps in the following areas: 
Policy Development 

o Gives you written bases tor policy development 

o Helps you learn about youi: district's program in an efficient and 

organized manner 
Community Involvement and Communication 

o Demonstrates to the community your involvement with the school program and 

promotes that program to the community 
o Allows the community to provide input as to its wishes for the educational 
, program through surveys, meetings, interviews and other methods of 

collecting evaluation information 
o Provides ready access to information when isolated complaints arise 
Staff Support 

o Demonstrates to the teachers and administrators your care and support for 
the curriculum and the work they are doing 

c 

o Helps the teaching staff show evidence to the community of their efforts 
o Helps you recognize the professional manner in which your district staff 

is performing, which wi^l build both your pride in their efforts and 

increase their pride and pleasure in their work 
Superintendent 

o Provides information for long-range planning for the curriculum 
o Helps to build cooperative working relationships between the board and 
superintendent 
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Co3ts and Cautions 

You need to be aware that many may, at the beginning , confuse program 
evaluation with a witch hunt or an effort to fire people* It should be clear 
that you are evaluating programs, not people. You, of course, do want to have 
an effective procedure for personnel evaluation, but do not confuse the two. 
Program evaluation may indicate staff development is needed, and your district 
will want to provide for staff improvement and opportunity for the staff to 
grow before any personnel action is taken, and that action would be the result 
of separate procedure. Program evaluation points out ways to improve 
program . Evaluation is a threatening procedure to all of us; therefore, care 
must be taken to be clear that the intent is to improve the curriculum for the 
students and that fair methods of evaluation are used. 

Other costs will include allocation of resources, such as time and money, 
to conduct evaluation and devoting board meeting time to consider curriculum 
and evaluation results. Is curriculum a high priority to your board? When 
you make decisions about program, particularly if you have to cut back, do you 
want these decisions to be in the best interest of your students? These 
issues would bear more discussion, but the main purpose of this article is to 
provide you with a process to begin (or improve) program evaluation in your 
district. 
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How Do We Get Started ? 

Assuming that you do not currently have a system of program evaluation 
with which you are pleased,* how do you start? Enthusiastic as you may be, 
you must realise that you cannot do everything at once. You must set 
priorities and realize limitations. Limitations include the number of 
personnel to carry out the evaluation, time, limited funds (Oh, yes, money!), 
and deadlines for decisions. 

One way to successfully deal with the limitations and help you start a 
program of quality evaluation is to set curricular priorities. The 
superintendent and school board can work together to decide which program is 
to receive priority for the first year. If you wish, you could do*a formal 
needs assessment, including information from many groups. You may have a 
pressing need of which you are acutely aware in your district and for which 
you feel you have justifiable reasons to designate that program as your top 
priority. In that case, you might decide not to start with a needs 
assessment, but to start immediately with an evaluation of that program. You 
may wish to make use of school site councils, parent advisory groups, 
curriculum committees or other organized groups to do a "short-cut" needs 
assessment while planning a more comprehensive needs assessment several years 
in the future. Remember the importance of long-range planning. 



*The authors would be very interested in hearing about evaluation systems 
currently used by the board members/super ii\tendents/readers of this article, 
particularly those which you feel are successful. Please send information to 
the authors in care of CSBA. 
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However you decide to set your priorities, keep your focus to one or, at 
most, two programs for the first time through the process. Expand to others 
as you have yoOr system of evaluation working well. Step number one, then, is 
to choose a program of emphasis . 

Asking the Critical Questions 
You have selected your program. Presumably, at the end of the evaluation 
process, you may be making a decision about that program; i.e., revise, 
continue as is, expand or cut back. Before the evaluation process starts, you 

must determine the questions you^ed answered to make that decision. A good 

) ' 

process for you to use is to first "brainstorm" and list all the questions you 
might like to have answered about the selected program. Depending on how your 
board operates, you may want to list questions separately first before 
discussing or you may want to formulate a list as a group through discussion. 
Your list may turn out to be very long and, no doubt, you will also have a 
good discussion among yourselves as you make this list. As you look at your 
list with pride and exhaustion, the problem of limitations is going to hit you 
again. There is probably no way that you can effectively get answers to all 
those questions (and hdvc anyone in the district with any energy left for 
other business) . Curb your enthusiasm enough to realize that you and your 
staff are goin<; to need stamina too. There are other programs worthy of 
attention, and this process needs to be ongoing, not just a one-time shot. In 
order to have a quality evaluation providing you with useful information, 
force yourselves to select critical questions for the focus of this program's 
evaluation, which questions/answers are absolutely vital to your decision 
about the program? In the curriculum institute session, we asked you to 
select three questions, but in your own district you may select one, two, 
three, or even four. 
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Some of the issues which will influence your selection may be: 

o What information already exists and has only to be taken off the 

shelf and dusted a bit? 
o Do you have personnel trained in evaluation methodology 'who can 

direct or carry out the evaluation? 
o Do you have committees or volunteers who could be tapped for help in 

doing surveys , interviews , etc.? 
o How much time do you have? 

o Are you willing and able to hire someone outside the district to 

conduct the evaluation? 
o Are there federal rules and guidelines for the program which will 

dictate that certain questions must be answered or that a certain 

methodology must be used in the evaluation plan? 

The Plan \ 

You will need to request an evaluation plan; this may come from district 
administrators , from an outside consultant to from whomever you/your 
superintendent designate (s) as responsible for the evaluation. The plan 
should include a timeline (you need information prior to your decision, not a 
week after) , and ways to collect information which will describe the program 
and information which can tell you the impact of the program or program 
results. An evaluation which gives a comparison of programs and the 
advantages/disadvantages of each can be very helpful. You will need to 
approve the plan and allocate the appropriate resources. You must either 
allow sufficient time or restrict the questions you want answered. Realize, 
too, that some questions cannot be answered conclusively as any of us would 
like. "I/we don't know" is sometimes a legitimate and the only honest 
answer. However, we keep trying to find answers. , 
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Final Steps 



When the evaluation process has been completed, you will receive a 
report. When designating the person (s) responsible for evaluation, the board 
and superintendent may want to indicate whether or not they expect 
recommendations in the final report. We suggest that it is the evaluator's 
responsibility to draw conclusions and make recommendations, acknowledging 
that the board and superintendent still have the option of accepting or 
rejecting those recommendations as they make their decision about the program. 

When you have received the report, you will want to discuss the results, 
then make y^ur program decision. You will also want to consider dissemination 
of information. For example, there will be some results you will want to 
publicize to your community, some- you will want to provide to the teachers 



within the program for their>use, some to the building administrators, and 
some to your curriculum specialists and/or central administration. 
At this atage, you will need to review: 

o Did we ask the right questions? Look at your policy on program 



o Is it adequate? If not, revise it now! 
Then, remembering that your purpose is to have a continuous, systematic 
evaluation of school programs, you select another program and start the 
process again. 

You will very likely find that your superintendent and administrative 
staff are very appreciative, helpful and welcoming of the board's interest and 
involvement. For your own district, you may wish to share some of the tasks 
in this article so that the superintendent is more or less involved, but that 
depends on the manner in which you usually work. The emphasis is, of course, 
on a cooperative, joint effort which will benefit the students in the district. 



evaluation. 
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Once Again , What is the Process the Board Can Use ?" 

Steps 

1. Set policy to insure program evaluation, 

2. Select program for evaluation. 

3. What decision faces the board at the conclusion of the evaluation? 

4. Brainstorm list of questions that need to be answered before that 
decision is made. 

5. Select (narrow down list to) a few questions vital to making your 
decision. 

6. Request an evaluation plan to answer those questions. 

7. Review plan provided by evaluator (should include timeline). 

8. Approve plan or request change. 

9. Allocate resources (time, money/ personnel). 
10. Monitor evaluation process as needed. 

U. Receive evaluation report. 

12. Discuss results. 

13. Make program decision. 

14. Provide results to the appropriate groups and individuals. 

15. Review your policy or policies. Revise if necessary. 

16 • Congratulate yourselves (and others) on a job well done; take a deep 
breath, and 

17. Select next program to be evaluated (start the process again). 
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Other Issues 

There are many other issues which you will encounter later, such as 
interpreting test results, deciding whether you have good quality and accurate 
information, gathering more specifics about what is realistic to expect from 
an evaluation, deciding how to deal with those resistant to evaluation of the 
program, and so on. You will find that some questions cannot be answered as 
completely as you wish. You may also want to know if you have an effective 
decision making process. Unfortunately, we do not have time to discuss all of 
the *ssues here. Instead, we have tried to provide you with some rationale 
for beginning or continuing program evaluation, a suggested process to begiin, 
some of the benefits and costs of program evaluation and, during the workshop 
session, some practice of that process and an opportunity to discuss with 
other board members how that process might work. We would hope to discuss 
other issues in future workshops or ar tides. A final note is that 
establishing a cooperative working relationship with your superintendent and- 
staff through a mutual goal is vital to a' successful program evaluation 
effort. In this process you can help your superintendent and management team; 
he/she/they can help you. Remember that your purpose is the one on which you 
campaigned: ^o~13prove^the^eduQajbional program and provide the best schools 
you can to your community. We wish you success in your efforts. 
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OBJECTIVES OF THE WORKSHOP 



To Participants 

Objectives of the workshop are to stimulate discussion on program evaluation 
to identify: 

1. Program evaluation and how is it used 

2. The school board's role in program evaluation 

3. Hew program evaluation can help school board members 

4. Some of the questions program evaluation can answer 

\ 

5. Some of the questions school board members need tb ask about program 
evaluation 




\ 





EXERCISE 1 
(5 minutes to complete) 

Where Are You? 



1. What percent of time does your board spend on curriculum or school 
programs? 



2. What percent of board time would you like to see your board spend on 
curriculum matters? 



3. Please state a question or concern about an existing school program in 
your district. 



4. What does evaluation mean to you? 
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EXERCISE 2 
(5 minutes to complete) 



Program Evaluation 



Please answer the following questions as they apply to your district* 

1. Which .teacher in your district has the 

most effective reading program? 



2, Would you vote to maintain your district's 
math program as it is now? 



3. Do students graduating from vocational 
courses have the skills needed for that 
vocation? 



4. Who is the best teacher in the district? 



5. Should students be required to take 
chemistry before graduating from 
high school? 
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EXERCISE 3 
(10 minutes to complete) 



Who is Responsible ? 



Following are some of the procedures that might be used with regard to 
curriculum evaluation. It is acknowledged that in many of these both school 
board and educators would be involved. However , for this exercise, you are 
• asked to make a choice according to who you think has the primary authority or 
responsibility for each item. 







Board 


process lonai 
Educators 


1. 


Ask questions about program 

and it's Avalnjihirtn * 

« 






2- 


program evaluation 






3. 


Develop a plan 






4. 


Modify/approve plan 






5. 


Establish priorities list along with 
program evaluation plan 






6. 


Make resources available for 
program evaluation 






♦ 

7. 


Carry out evaluation plan 






8. 


Report evaluation results 






9. 


Use evaluation results 






10. 


Advise community of evaluation 






11. 


Monitor use of the evaluation plan 







EXERCISE 4 
The Boards Role in Program Evaluation 

1. Raise questions and concerns about curriculum 

2. Set a policy for a systematic plan of curriculum evaluation 

3. Prioritize evaluation efforts 
A. Receive evaluation results 

5, Use results for decision making 

6, Use , evaluation information to select criteria as basis of decisions 
about curriculum 

1. Write those criteria and let community know on what criteria a 
decision was made 

8 # Communicate with the community in appropriate ways about evaluation 

9* Monitor implementation of evaluation plan/policy ( remember , it is a 
continuous f ongoing process) 

These should be done in cooperation with the superintendent (and other 
educators, as appropriate). 



Can you add to this list? 
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EXERCISE 5 

How Would a School Board Pse Information From Evaluation? 



1. 
2. 
3. 
4. 
5. 
6. 
7. 
8. 
9. 
10. 
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EXERCISE 6 
(15 minutes) 

Program Analysis 



Program Selected , 

\ 

Using the situation for your group> follow the three steps. 
1. Identify the decision you think the board should make: 



2. List questions that need to be asked: 



3. Select the three most important questions. 



4. List three pieces of evaluation information you will need from the 
superintendent* 



SOME EVALUATION OPTIONS 



Who Might Conduct the Evaluation? 



Internal 

Evaluation staff 
Subject area specialists 

Special committee (specify composition: teachers? administrators? nurses? a 
combination? parents?) 

Existing coiaaittee (specify: math curriculum committee , etc.) 

c 

External 

Individual evaluator 
Evaluation team 
Parent coianittee study 



What Types of Data Might Be Collected? 



Sur v ey oft 

Ccxaaunity, students, teachers,* administrators , graduates (former students), 
classified staff, etc. 



For Test Data: 

That compares students with other students across nation, request 
norm- referenced test data 

That shows student growth according to objectives of district curriculum, 
request objective-referenced tosts 

That shows student performance according to 3et standards, such as 
psycho-motor skills in P.E., request performance tests 

That shows student scores on teacher-made tests, request teacher-constructed 
test data 
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Other Data 



Objectives 

Cost information 

Similar information about two programs for comparison purposes 
Strictly controlled research information 

For Information About a Program : 

To help improve the program as it is being developed and tried, request 
f ormative evaluation. 

To su mm a r ize results at end of a program or a year, request summative 
evaluation. 

School Board Action 

h recent study of schools in California found that in the most effective 
schools the evaluation and testing programs were a part of the policy and 
procedures of the district. 
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DEFINITIONS 



CURRICULUM 

The Actual Experiences Provided Pupils by the School 
to Realize Goals 

Some Influences on those Experiences: 

Students 

Teachers 

Administrators 

Superintendent 

Materials 

Community 

School Board 

EVALUATION: 

Evaluation is a Two Step Proces. 

1. Identify and collect information 

2. Judge and place value on that information so 
that a decision can be made 

PROGRAM EVALUATION LOOKS AT: 

• Description of the program 

• Impact of the program 
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RESOURCE LIST 



Keys to School Boardsmanship . Portland, Oregon: Northwest Regional 

Educational Laboratory, 300 S. W. Sixth Avenue, Portland, Oregon 97204. 
(503) 248-6800 

National School Boards Association, 1055 Thomas Jefferson, N. W. , Suite 600, 
Washington, C. C. 20007. (202) 337-7666 

Alaska Association of School Boards, 204 North Franklin Street, Suite 3, 
Juneau, Alaska 99801. (907) 586-1083 

Idaho School Boards Association, 427 Idaho Building, P. 0. Box 2577, Boise, 
Idaho 83701. (208) 342-6441 

Montana School Boards Association, 501 North Sanders, Helena, Montana 59601. 
(406) 442-2180 

Oregon School Boards Association, P. 0. Box 1068, Salem, Oregon 97308. (503) 
588-2800 

Washington State School Directors' Association, 200 East Union Avenue, Olympia, 
Washington 98501c (206) 753-3305 

Joint Committee on Standards for Educational Evaluation, Standards for 

Evaluations of Educational Programs , Projects and Materials . Draft, 1978. 

Ovens, T. R. Program Evaluation for Busy Administrators . Portland, Oregon: 
Northwest Regional Educational Laboratory, 1979. 
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Here's the situation: 



Math 



Your board is taking a look at math requirements at secondary level. There is 
a geometry class offered at ninth grade (junior high) level for students who 
were eligible to take algebra in eighth grade. The students are selected from 
the results of a test administered^St the end, of seventh grade , One of the 
members of your board has an extreme dislike for the teacher of that, junior 
high geometry class and suggests that it be discontinued, Last^week five 
parents of students in the geometry class attended the board meeting to voice 
their support for the class, feis year only 20 students were eligible and 
signed up for the class. 

As a board, 

1. Identify the decision which needs to be made. (Just identify, don't 
make the decision yet!) 

2. List the questions you want to ask, 

3. Select the three most important questions. 
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Here's the situation: 



Foreign Language 

< - / 

For several years your district has had a foreign language progranr' in the 
middle school, seventh and eighth graces, in which French, Spanish and German 
are taught* Two high school teachers have complained that the programs taught 
in the middle school prevent students from taking their classes in high 
school* A check reveals that enrollment in French and Spanish has declined at 
^fcigt^ school level* Keeping three foreign languages in the middle school is 
expensive, particularly since only five students signed up for Gennan IV 
(fourth year), at high school for the past two years. Your board just received 
a letter supporting the middle school program signed by 10 parents whose 
children have participated .In the middle school foreign languages classes* 

You accidentally overheard two teachers discussing the program; they said that 
the reason enrollment declines in high school is that the high school doesn't 
have a\ strong or interesting program; that it concentrates on reading 
literature instead of teaching students how to speak the language* (Neither^ 
of these teachers teach language*) 

\ 

As a board, 

1. Identify the decision which needs to be made. (Just identify, don't 
make the decision yet I) 

2. List the questions you want to ask. 

3. Select the three most important questions. 
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Here's the situation: 



Social Studies 




A social studies teacher in a suburban junior high on the east coast takes 
students into a slum area of the city for them to see poverty living 
conditions first hand* A board member objects to this field trip because the 
next door neighbor's child came home crying about the conditions seen on the 
trip* The board member does not object to students reading about conditions 
of poverty/ but does object to such a graphic way of demonstrating the 
situation. 

As a board, 

1. Is this a curriculum issue? Why or why not? 

2. Does the board have a decision to make? If so, what is the 
decision? (Just identify, don't make the decision yetl) 

3. if there is a board decision, list questions you would want to ask, 

4. Select the three most important questions. 



\ 




\ 



7b 



14 



E'Xe's the situation: 



Talented and Gifted 

Your board is considering a gifted and talented program. Your district's 
community is composed mainly of refinery workers, farmers and college 
professors. About one third of your enrollment cjre minority students (about 
20 percent black, 10 percent Hispanic). Some members of the community are 
dissatisfied with the district's efforts to improve achievement level of two 
of the elementary schools which have had consistently lower scores than other 
schools. Other community people affiliated with the college have spoken out 
that not enough attention is given to college bound students in the district. 
District enrollment has declined slightly over the past two years. 

As a board, 

1. Identify the decision which needs to be made. (Just identify, don't 
make the decision yet I) 

2* List the questions you want to ask. 

3* Select the three most important questions. 
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Here's the situation: 



Bilingual Education 



About 30 percent of your community is Spanish speaking; about 5 percent speak 
German in the home. The foreign language program teaches French , German, 
Russian and Spanish in high school only. 

You have been requested to install a maintenance program of bilingual 
education by one of your building administrators, who has written a proposal. 
The proposal, if accepted, would be funded by the federal government the first 
year, but would be funded by the district the second and third years (half the 
second year and total district funding the third year). 

Your task is: 
As a board, 

1. Identify the decision which needs to be made. (Just identify, don't 
make the decision yeti) 

2* List the questions you want to ask. 

3. Select the three most important questions. 
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Here's the situation: 



Sex Education 



Your district currently has no formal or planned sex education program. 
However , it appears several teachers are acting on their own* 

Your community is split in half on whether to have a sex education program 
according to a survey conducted last year . Some members of the community feel 
very ptrongly that sex education belongs in the home, not in the schools* 
Another group feels equally as strong that the schools need a sex education 
program. There were several pregnancies last year at high school level and 
two- at junior high* You've had a number of personal calls both for and 
against the issue. Community pressure is building for some type of decision. 

As a board, 

1. Identify the decision which needs to be made. (Just identify, don't 
make the decision yet I) 

2* List the questions you want to ask. 

3. Select the three most important questions.^ 
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Here's the situation: 



Read ing 



There are-, four elementary schools in your district. One has had an 
individualized reading program for three* years; the other three have a program 
which is more traditional and has been used in the district for 10 years. 
From testing conducted regularly in the district , you see no achievement 
differences between the two programs. The individualized reading program is 
more expensive to operate* A neighbor of yours is a teacher's aide in the 
individualized reading program and tells you often how great it is. A friend, 
whom you usually see at church on Sundays , and whose opinion you generally 
respect, tells you that the traditional program is much better. 

As a board, 

1. Identify the decision which needs to be made. (Just identify, don't 
make the decision yetl) 

2. List the questions you want to ask. 

3» Select the three most important questions. 
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Process For Chan ge 

Evaluating Your Instructional Programs 

Dr. BETTY TOMBLIN 
Dr. LESLIE WOLFE 

Authors' biographies on page 79 

collected in a systematic way from the appropriate 
group as well as information from your informal 
sources? 

At times an angry parent or community member 
will demand you take action; you may react before 
you find out if that one concerned person is 

CALIFORNIA SCHOOL BOARDS 



Every time you respond to a question about 
_ your curriculum, you are making an evalua- 
tion: First, you collect Ci* identify information about 
the program, second, you judge or place a value on 
the worth of that program. You need to consider 
your sources of information. Do you have data 
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expressing a comment representative of others or if 
it is an isolated incident: Do you have an iceberg — 
or an ice cube? A complaint may represent a small 
incident that can be solved between the persons 
involved or by providing information or it may 
indicate a real problem within the curriculum or 
instructional program. Ongoing systematic evalua- 
tion can help you determine which isthecase.lt will 
help you be a leader that takes a pro-active, not a 
reactive, role. 

What Is Evaluation? 

Evaluation is an important part of the ongoing 
process to contir.ually improve the curricular pro- 
gram* Phases of that cycle are planning, implementa- 
tion, evaluation, revision; and the cycle begins 
again. 

The systematic and continuous evaluation of 
programs should be monitored by the board. The 
board's role is primarily to establish the philosophy 
and goals for the district, set the policy for program 
evaluation, request evaluation, use evaluation infor- 
mation for its program decisions, and examine 
congruence between programs and philosophy. 

How Can It Help You? 

The payoff for boards, students, teachers and 
administrators can be tremendous. In addition to 
helping boards decide whether to adopt, expand, or 
cut back programs, evaluation helps in the following 
areas: 

Po//cy Development 

• Gives you written bases 

• Helps you learn about your district's program 

in an efficient manner 
Community Involvement and Communication 

• Demonstrates to the community your involve- 

ment with the school program and promotes 
. that program to the community. 

• Allows the community to provide input through 

surveys, meetings, interviews and other methods. 

• Provides ready access to information when 

isolated complaints arise. 
Staff Support 

• Demonstrates to the teachers and administra- 

tors your care and support for the curriculum 
and the work they are doing. 

• Helps the teaching staff show evidence to the 

community of their efforts. 
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'The governing board of every school 
district shall evaluate 
~ks educational program, and shall make 
such revisions as it deems necessary/' 

Education Code 



• Helps you recognize the professional manner 

in which your district staff is performing. 
Superintendent 

• Provides information for long-range program 

planning. 

• Helps buijd cooperative working relationship 

between the board and superintendent. 

How Do We Get Started? 

Assuming that you do not currently have a 
system of program evaluation with which you are 
pleased, how do you start? 1 Enthusiastic as you may 
be,' you can not do everything at once. You must set 
priorities, and realize such limitations as the number 
of personnel and amount of time available to carry 
out the evaluation, limited funds, and deadlines for 
decisions. 

One way to deal with such limitations at the start 
is to establish priorities. The superintendent and 
school board can work together to choose the 
program to receive priority the first year. The board 
might choose to do a formal needs assessment to 
obtain information from many groups, or your 
district may have an obvious and pressing need that 
makes the choice for you; in that case you might 
start immediately with an evaluation of that program. 
If the need is somewhat less pressing, you may 
involve school site councils, parent advisory groups, 
curriculum committees or other organized groups 
in doing a "short cut" needs assessment while 
planning a more comprehensive needs assessment 
several years in the future. 

However you decide to set your priorities, keep 
your focus to one or at most two programs for the 
first time through the process. 

(continued on page 38) 



} The authors would be interested in hearing about evalua- 
tion systems currently used by board members/super- 
intendents, particularly those which you believe are 
successful. 
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t Evaluating Program, continued from page 35) 

Asking the Critical Questions 

You have selected your program. Before t u e 
evaluation process st?r.ts, you must determine the 
questions you need answered. 

One approach is for the board to "brainstorm" 
to come up with all the questions you want answered 
about the seleaed program. You can either list 
quesl.ons individually before discussing, or you may 
make up a liJt as a group through discussion. 

As you review the list, thaproblem of limitations 
is going to hit you again: There is probably no way 
that you can get answers to ail those questions and 
have anyone in ihe district with any energy left for 
other business. As you face cutting the list, bear in 
frimd this process will be ongoing, not just a one- 
time shot. Force yourselves to select critical questions 
tor the focus of this initial evaluation. Among these 
might be 

• Which questions/answers are absolutely vital to 
your decision about the program? 

• What information exists and has only to be taken 
off the shelf and dusted off? 

• Do you have personnel trained in evaluation 
methodology who can direct or carry out the 
evaluation*' 

• Do you have committees or volunteers who 
could be tapped for help in doing surveys, 
interviews, etc? 

• How much time do you have? 

• Are you willing and abletohirt: "neoneoutside 
the district to conduct the evaluation? 

• Are there federal rules and guidelines for the 
program which will dictate that certain questions 
must be answered or a certain methodology 
must be used in the evaluation plan? 



The Plan 

An evaluation plan must be designed at your 
request by district administrators, by an outside 
consultant or by whomever you and the superin- 
tendent designate as responsible for the evaluation. 
The plan should include a timeline and a discussion 
of ways to collect information which will describe 
the program itself and the impact of the program or 
program results. An evaluation plan which will 
compare programs and the advantages/disadvan- 
tages of each can be very helpful. 

Final Steps 

When the evaluation process has been com- 
pleted, the board will receive a report from the 
person assigned to conduct the evaluation. The 
evaluator should be required to draw conclusions 
and make recommendations; your board and super- 
intendent may, of course, elect to accept pr reject those 
recommendations in deciding about the program. 

When you have received the report and made 
your program decision, you will also want to con- 
sider dissemination of information: you will want to 
publicize some results to your community; some 
you will want to provide to the teachers, some to the 
building administrators, some to your curric ^m 
specialists and/or central administration. 

Now is the time for review. Did you ask the right 
questions? Is your policy on program evaluation 
adequate? If not, revise it NOW! Then, remembering 
that your purpose is to have a continuous, systematic 
evaluation of school programs, select another pro- 
gram and start the process again. 

You will very likely find that your superintendent 
and administrative staff are very appreciative, helpful 
and welcoming of the board's interest and involve- 
ment. The emphasis is on a cooperative joint effort 
which will benefit the students in your district. ■ 

> 
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THE SCHOOL BOARD'S ROLE IN PROGRAM EVALUATION 



Workshop Critique 



In order to improve the workshop, we would appreciate your time in answering 
the folding questions. Feel free to make comments regarding any item* 



1. Did this workshop cover what it was advertised to cover? 



No 
1 



Somewhat 
3 



2 3 . 4 

If rot, what was missing that ycu expected to cover? 



Yes 
5 



2. 



Do you feel that the content of this workshop will be of help to you as a 
school board member? 



No 
1 



Somewhat 
3 



Yes 
5 



Is there one (or more) specific action (s) you can take back to your 
district concerning program and/or its evaluation? 



Definitely 
, Not 
1 



Somewhat 
3 



Definitely 
Yes 
5 



Can you name one? 



4. Was adequate time allowed for questions and discussion during the 
presentation? 



No 
1 



Somewhat 



-Yea. 



5* Please state one thing you learned or gained from the session: 
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6. What questions remain unanswered for you regarding program evaluation? 



7. Relative to workshops you have attended, how do you rate this workshop? 

2 3 4 



Very Poor 
1 



Very Good 
5 



8. How would you rate the overall performance of the leader — knowledge and 
presentation? 



Very Poor 
1 



Very Good 
5 



9. Are you: 



A school board member 
A superintendent 
Other: please state: 



10. What can ths leader do to improve the session? 



11.- Any additional comments for improving this and future workshops will be 
appreciated. 
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RELATED TRANSPARENCIES 
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PROGRAM EVALUATION 
WORKSHOP OBJECTIVES 



Program evaluation and how is it used 
The school board's role in program evaluation 
Evaluation can help school board members 
Questions program evaluation can answer 
Questions school board members need to ask 
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USE OF BOARD TIME 

10% or less 

10% to 20% 

20% to 30% 

30% to 40% 

40% to 50% 

50% to 60% 

60% to 100% 
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WISH IT WERE 

0% - 25% 
25% - 50% 

J 

50% - 75% 



Goals 
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• Academic goals 

• Social goals 

• Vocational goals 

• Personal development goals 
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Resources 

• Textbooks 
® Teachers 

• Principais 

• Sjperiniendents 

• Teacher aides 

• Custodial staff 

• Counselors 

• Others in the school 

• Learning materials 

• Physical facilities 

• Etc. 
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CONE OF EXPERIENCES* 




•3 

*Edgar, Dale. Audiovisual methods in teaching. Revised edition. 
New York: Dryden Press, 1954. pg. 43. 
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LEARNING PROGRAM 

THE ACTUAL EXPERIENCES PROVIDED 
PUPILS BY IriE SCHOOL TO ACHIEVE 
GOALS 
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No Control 

The abilities of students 
Parent expectations 
Community expectations 
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Influences on Learning 

• Textbooks 

• Instructional Materials 

• Teachers 

• Principals 

• Time on Task 



So 



Judgments based on 

Do you have good information? 
Where did you get your information? 
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Types of Information 

• Intuition 

• Personal likes/dislikes 

• Opinion of neighborhood 

• Test data 

• External evaluation team 

• Winning state championship 

• New reading program 

• Research report 

• Report by specialist 

• Advisory committee report 
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TIP OF THE ICEBERG VS ICE CUBE 



When you hear a complaint or a question, you 
don't know if you have a single ice cube or 




UNLESS YOU HAVE A PLANNED SYSTEM OF EVALUATION 
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PROGRAM EVALUATION IS A TWO STEP PROCESS 

1. Identify and collect information 

2. Judge and place value on that information so 
that a decision can be made 
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The School Board has a Responsibility for 
Providing a Program of Quality Instruction 
Within Available Resources 

Their Responsibility is Ongoing and Continuous. 
The School Board must Continue to ask Questions 
and Seek Information about the curriculum. . 

Through a Systematic Plan for Curriculum 
Evaluation 
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Questions the board must ask 



• Is the curriculum sound? 

• Is the staff implementing the plan? 

• Are the students learning? 
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CYCLE OF PROGRAM EVALUATION 




THE SCHOOL BOARD'S ROLE IN CURRICULUM EVALUATION 

• Raise questions and concerns about curriculum 

t Set policy for a systematic plan of curriculum evaluation 

• Receive evaluation results 

• USE evaluation information to select criteria as basis of 
decisions about curriculum 

• WRITE those criteria and let community know on what 
criteria a decision was made 

• Communicate with the community in appropriate ways about 
evaluation 

• Monitor implementation of evaluation plan/policy (remember, 
it is a continuous, ongoing process) 

PE 
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USES OF EVALUATION 



• District funding of program 

• Continuation pf a program 

• Change of a program 

• Basis for a decision 

• Communicate with community 

• Examine congruence 

• Basis for policy 

• Meet federal and state guidelines 

• Demonstrate needs of programs 

• Adoption of new program 
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School board members are accountable 
and responsible for the 
quality of the learning program: 

• Evaluation of teachers 

• Evaluation of principals 

• Evaluation of superintendents 

• Evaluation of instructional materials 

• Evaluation of student achievement 

—Thomas Shannon 
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DEFINITIONS 



CURRICULUM 

The Actual Experiences Provided Pupils by the School 
to Realize Goals 

Some Influences on those Experiences: 

Students 

Teachers 

Administrators 

Superintendent 

Materials 

Community 

School Board 

EVALUATION: 

Evaluation is a Two Step Process 

1. Identify and collect information 

2. Judge and place value on that information so 
that a decision can be made 

PROGRAM EVALUATION LOOKS AT: 
t Description of the program 
• Impact of the program PE 20 
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